Method and apparatus for generating information signal to be recorded

ABSTRACT

A compressively-encoding-resultant video signal resulting from compressively encoding a video signal, a first compressively-encoding-resultant audio signal resulting from compressively encoding a first audio signal having a synchronized relation with the video signal, a reference clock information signal relating to synchronization between the compressively-encoding-resultant video signal and the first compressively-encoding-resultant audio signal, and an indication timing information signal designating an indication timing of the video signal are multiplexed to get an AV multiplexing-resultant signal. A second audio signal is compressively encoded to get a second compressively-encoding-resultant audio signal. A sound-production-timing information signal is added to the second compressively-encoding-resultant audio signal to get a substitution playback audio signal. During playback, the substitution playback audio signal can be reproduced instead of the first audio signal while being synchronous with the video signal. The sound-production-timing information signal responds to the reference clock information signal, and relates to synchronization with the video signal.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to a method of generating an information signalto be recorded, a method of reproducing an information signal, a methodof transmitting an information signal, an apparatus for generating aninformation signal to be recorded, an apparatus for reproducing aninformation signal, an apparatus for transmitting an information signal,an information-signal recording medium, and a computer program fortransmitting an information signal.

2. Description of the Related Art

In some cases, an analog audio signal and an analog video signalaccompanied with the analog audio signal are converted into a digitalaudio signal and a digital video signal before being transmitted orrecorded on a recording medium. The conversion of the analog videosignal into the digital video signal is based on, for example, acompressively encoding procedure. A temporal error tends to occurbetween the digital audio signal and the digital video signal.

According to a known system designed to compensate for such a temporalerror between audio and video, first timing information relating todesired audio playback moments is added to a digital audio signal whilesecond timing information relating to desired video playback moments isadded to a digital video signal. The desired audio playback moments andthe desired video playback moments are chosen to provide synchronizationbetween audio and video. During playback, information represented by thedigital audio signal is converted into sound at a timing decided by thefirst timing information while information represented by the digitalvideo signal is indicated at a timing decided by the second timinginformation. As a result, the playback of audio and the playback ofvideo are synchronized with each other.

In a known multiplexing-based transmitting or recording system, aportion of a digital audio signal and a portion of a digital videosignal which relate to a same desired playback moment are transmitted orrecorded while being placed in a same group through a multiplexingprocedure. This design makes possible the synchronous playback of audioand video.

Recently, multiplexed data containing compressively-encoded video dataand audio data have sometimes been handled as a form of a bit streamwhich is referred to as a transport stream or a program stream, andwhich is designed to make possible the synchronous playback of audio andvideo. Generally, it is not easy to edit only audio data in suchmultiplexed data.

An example of the editing is post-recording or after-recording whichedits only audio data in multiplexed data (a bit stream) and replacesthe audio data with new data. The post-recording (the after-recording)is complicated.

Japanese patent application publication number 6-275018/1994 discloses arecording and reproducing apparatus which operates in either a recordingmode or a reproducing mode. During the recording mode of operation ofthe apparatus in Japanese application 6-275018, information signals fromN sources are converted into first digital signals through acompressively encoding procedure, and the first digital signals aremodulated and converted into second digital signals. The second digitalsignals are written into a recording medium on a time sharing basis.During the reproducing mode of operation, the second digital signals areread out from the recording medium on a time sharing basis, and theread-out second digital signals are demodulated and converted back intothe reproduced first digital signals. The reproduced first digitalsignals are converted back into the reproduced information signals. Thereproduced information signals are simultaneously outputted.

Japanese patent application publication number 11-144378/1999 disclosesa method of after-recording (post-recording) in which original dataincluding a video bit stream are read out from a digital recordingmedium. The read-out original data are decoded. Audio data are encodedinto a new audio bit stream in synchronism with the decoding of theoriginal data. The new audio bit stream is written into an area of thedigital recording medium which approximately corresponds in timeposition to the original-data recording area.

Japanese patent application publication number P2000-197005A disclosesan information recording medium having an area “A” and an area “B”. Thearea “A” stores a stream of packs including video packs and audio packs.The area “B” stores a table having first, second, and third informationpieces. By referring to the first information piece in the table, adecision is made as to whether or not audio data in the audio packs inthe area “A” correspond to silence. In the case where the audio data inthe audio packs correspond to silence, post-recording (after-recording)can be implemented as follows. By referring to the second and thirdinformation pieces in the table, a new audio signal is encoded into newaudio data, and the new audio data are formatted into new audio packs.The new audio packs are written over the old audio packs in the area“A”.

Prior-art post-recording (prior-art after-recording) writes new audiodata into an original multiplexed main stream so that the originalmultiplexed main stream changes to new one. Generally, it is difficultto convert the new multiplexed main stream back into the original one.

In the prior art, it is difficult to post-record a plurality of newaudio signals, and to select one from the new audio signals duringplayback.

SUMMARY OF THE INVENTION

It is a first object of this invention to provide a method of generatingan information signal to be recorded which makes possible thepost-recording of a plurality of new audio signals, the selection of onefrom the new audio signals during playback, and the synchronous playbackof audio and video.

It is a second object of this invention to provide an improved method ofreproducing an information signal.

It is a third object of this invention to provide an improved method oftransmitting an information signal.

It is a fourth object of this invention to provide an improved apparatusfor generating an information signal to be recorded.

It is a fifth object of this invention to provide an improved apparatusfor reproducing an information signal.

It is a sixth object of this invention to provide an improved apparatusfor transmitting an information signal.

It is a seventh object of this invention to provide an improvedinformation-signal recording medium.

It is an eighth object of this invention to provide an improved computerprogram for transmitting an information signal.

A first aspect of this invention provides a method of generating aninformation signal to be recorded. The method comprises the steps ofmultiplexing 1) a compressively-encoding-resultant video signal whichresults from compressively encoding a video signal, 2) a firstcompressively-encoding-resultant audio signal which results fromcompressively encoding a first audio signal having a synchronizedrelation with the video signal, 3) a reference clock information signalrelating to synchronization between the compressively-encoding-resultantvideo signal and the first compressively-encoding-resultant audiosignal, and 4) an indication timing information signal based on thereference clock information signal and designating an indication timingof the video signal to get an AV multiplexing-resultant signal;generating a second audio signal as a substitution playback audio signalwhich can be reproduced instead of the first audio signal while beingsynchronous with the video signal during playback; compressivelyencoding the second audio signal to get a secondcompressively-encoding-resultant audio signal; and adding asound-production-timing information signal to the secondcompressively-encoding-resultant audio signal to get the substitutionplayback audio signal, the sound-production-timing information signalbeing based on the reference clock information signal and relating tosynchronization with the video signal.

A second aspect of this invention is based on the first aspect thereof,and provides a method further comprising the steps of dividing thesubstitution playback audio signal into packets each having a prescribeddata size, and multiplexing the reference clock information signal andthe sound-production-timing information signal with each of the packets.

A third aspect of this invention is based on the first aspect thereof,and provides a method wherein the substitution playback audio signalcomprises a plurality of subordinate substitution playback audio signalswhich result from compressively encoding audio signals respectively, anddifferent identification signals are added to the subordinatesubstitution playback audio signals respectively.

A fourth aspect of this invention provides a method of reproducing aninformation signal. The method comprises the steps of reproducing an AVmultiplexing-resultant signal generated by multiplexing 1) acompressively-encoding-resultant video signal resulting fromcompressively encoding a video signal, 2) a firstcompressively-encoding-resultant audio signal resulting fromcompressively encoding a first audio signal having a synchronizedrelation with the video signal, 3) a reference clock information signalrelating to synchronization between the compressively-encoding-resultantvideo signal and the first compressively-encoding-resultant audiosignal, and 4) an indication timing information signal based on thereference clock information signal and designating an indication timingof the video signal; reproducing the video signal from the reproduced AVmultiplexing-resultant signal; reproducing a substitution playback audiosignal to reproduce a second audio signal, the substitution playbackaudio signal being generated by 1) compressively encoding the secondaudio signal to get a second compressively-encoding-resultant audiosignal, and 2) adding a sound-production-timing information signal tothe second compressively-encoding-resultant audio signal, thesound-production-timing information signal designating a soundproduction timing of the second compressively-encoding-resultant audiosignal; deriving the reference clock information signal and theindication timing information signal from the reproduced AVmultiplexing-resultant signal; deriving thecompressively-encoding-resultant video signal from the reproduced AVmultiplexing-resultant signal; decoding the derivedcompressively-encoding-resultant video signal to get adecoding-resultant video signal; supplying the decoding-resultant videosignal as an output video signal in response to the derived referenceclock information signal and the derived indication timing informationsignal; deriving the sound-production-timing information signal and thesecond audio signal from the reproduced substitution playback audiosignal; supplying the derived second audio signal as an output audiosignal in response to the derived reference clock information signal andthe derived sound-production-timing information signal; comparing thereference clock information signal and the indication timing informationsignal to get a first comparison result; supplying the video signal asan indication-purpose signal in accordance with the first comparisonresult; comparing the reference clock information signal and thesound-production-timing information signal to get a second comparisonresult; and supplying the second audio signal signal as asound-production-purpose signal in accordance with the second comparisonresult.

A fifth aspect of this invention is based on the fourth aspect thereof,and provides a method wherein the substitution playback audio signalcomprises a plurality of subordinate substitution playback audio signalswhich result from compressively encoding source audio signalsrespectively, and different identification signals are added to thesubordinate substitution playback audio signals respectively, andwherein one of the source audio signals is selected by referring to theidentification signals, and the selected source audio signal and thevideo signal are supplied as output reproduced signals synchronous witheach other.

A sixth aspect of this invention provides a method of transmitting aninformation signal. The method comprises the steps of generating a mainAV multiplexing-resultant signal by multiplexing 1) acompressively-encoding-resultant video signal which results fromcompressively encoding a video signal, 2) a firstcompressively-encoding-resultant audio signal which results fromcompressively encoding a first audio signal having a synchronizedrelation with the video signal, 3) a reference clock information signalrelating to synchronization between the compressively-encoding-resultantvideo signal and the first compressively-encoding-resultant audiosignal, and 4) an indication timing information signal designating anindication timing of the video signal on the basis of the referenceclock information signal; generating a substitution playback audiosignal by 1) compressively encoding a second audio signal to get asecond compressively-encoding-resultant audio signal, and 2) adding asound-production-timing information signal to the secondcompressively-encoding-resultant audio signal, thesound-production-timing information signal designating a soundproduction timing of the second compressively-encoding-resultant audiosignal on the basis of the reference clock information signal, whereinthe second audio signal can be reproduced instead of the first audiosignal; generating a substitution AV multiplexing-resultant signal as asignal for reproducing the compressively-encoding-resultant video signaltogether with the second compressively-encoding-resultant audio signal;feeding the substitution AV multiplexing-resultant signal to atransmission line; deriving the second compressively-encoding-resultantaudio signal from the substitution playback audio signal; and replacingthe first compressively-encoding-resultant audio signal in the main AVmultiplexing-resultant signal with the derived secondcompressively-encoding-resultant audio signal to change the main AVmultiplexing-resultant signal into the substitution AVmultiplexing-resultant signal before feeding the substitution AVmultiplexing-resultant signal to the transmission line.

A seventh aspect of this invention provides an apparatus for generatingan information signal to be recorded. The apparatus comprises means formultiplexing 1) a compressively-encoding-resultant video signalresulting from compressively encoding a video signal, 2) a firstcompressively-encoding-resultant audio signal resulting fromcompressively encoding a first audio signal having a synchronizedrelation with the video signal, 3) a reference clock information signalrelating to synchronization between the compressively-encoding-resultantvideo signal and the first compressively-encoding-resultant audiosignal, and 4) an indication timing information signal based on thereference clock information signal and designating an indication timingof the video signal to get an AV multiplexing-resultant signal; meansfor generating a second audio signal as a substitution playback audiosignal which can be reproduced instead of the first audio signal whilebeing synchronous with the video signal during playback; means forcompressively encoding the second audio signal to get a secondcompressively-encoding-resultant audio signal; and means for adding asound-production-timing information signal to the secondcompressively-encoding-resultant audio signal to get the substitutionplayback audio signal, the sound-production-timing information signalbeing based on the reference clock information signal and relating tosynchronization with the video signal.

An eighth aspect of this invention provides an apparatus for reproducingan information signal. The apparatus comprises means for reproducing anAV multiplexing-resultant signal generated by multiplexing 1) acompressively-encoding-resultant video signal which results fromcompressively encoding a video signal, 2) a firstcompressively-encoding-resultant audio signal which results fromcompressively encoding a first audio signal having a synchronizedrelation with the video signal, 3) a reference clock information signalrelating to synchronization between the compressively-encoding-resultantvideo signal and the first compressively-encoding-resultant audiosignal, and 4) an indication timing information signal designating anindication timing of the video signal; means for reproducing the videosignal from the reproduced AV multiplexing-resultant signal; means forreproducing a substitution playback audio signal to reproduce a secondaudio signal, the substitution playback audio signal being generatedby 1) compressively encoding the second audio signal to get a secondcompressively-encoding-resultant audio signal, and 2) adding asound-production-timing information signal to the secondcompressively-encoding-resultant audio signal, thesound-production-timing information signal designating a soundproduction timing of the second compressively-encoding-resultant audiosignal, wherein the second audio signal can be reproduced instead of thefirst audio signal; means for deriving the reference clock informationsignal and the indication timing information signal from the reproducedAV multiplexing-resultant signal; means for deriving thecompressively-encoding-resultant video signal from the reproduced AVmultiplexing-resultant signal; means for decoding the derivedcompressively-encoding-resultant video signal to get adecoding-resultant video signal; means for supplying thedecoding-resultant video signal as an output video signal in response tothe derived reference clock information signal and the derivedindication timing information signal; means for deriving thesound-production-timing information signal and the second audio signalfrom the reproduced substitution playback audio signal; and means forsupplying the derived second audio signal as an output audio signal inresponse to the derived reference clock information signal and thederived sound-production-timing information signal.

A ninth aspect of this invention is based on the eighth aspect thereof,and provides an apparatus wherein the substitution playback audio signalcomprises a plurality of subordinate substitution playback audio signalswhich result from compressively encoding source audio signalsrespectively, and different identification signals are added to thesubordinate substitution playback audio signals respectively, andwherein one of the source audio signals is selected by referring to theidentification signals, and the selected source audio signal and thevideo signal are supplied as output reproduced signals synchronous witheach other.

A tenth aspect of this invention provides an apparatus for transmittingan information signal. The apparatus comprises means for generating amain AV multiplexing-resultant signal by multiplexing 1) acompressively-encoding-resultant video signal which results fromcompressively encoding a video signal, 2) a firstcompressively-encoding-resultant audio signal which results fromcompressively encoding a first audio signal having a synchronizedrelation with the video signal, 3) a reference clock information signalrelating to synchronization between the compressively-encoding-resultantvideo signal and the first compressively-encoding-resultant audiosignal, and 4) an indication timing information signal designating anindication timing of the video signal on the basis of the referenceclock information signal; means for generating a substitution playbackaudio signal by 1) compressively encoding a second audio signal to get asecond compressively-encoding-resultant audio signal, and 2) adding asound-production-timing information signal to the secondcompressively-encoding-resultant audio signal, thesound-production-timing information signal designating a soundproduction timing of the second compressively-encoding-resultant audiosignal on the basis of the reference clock information signal, whereinthe second audio signal can be reproduced instead of the first audiosignal; means for generating a substitution AV multiplexing-resultantsignal as a signal for reproducing the compressively-encoding-resultantvideo signal together with the second compressively-encoding-resultantaudio signal; means for feeding the substitution AVmultiplexing-resultant signal to a transmission line; means for derivingthe second compressively-encoding-resultant audio signal from thesubstitution playback audio signal; and means for replacing the firstcompressively-encoding-resultant audio signal in the main AVmultiplexing-resultant signal with the derived secondcompressively-encoding-resultant audio signal to change the main AVmultiplexing-resultant signal into the substitution AVmultiplexing-resultant signal before feeding the substitution AVmultiplexing-resultant signal to the transmission line.

An eleventh aspect of this invention provides an information-signalrecording medium having a common surface storing both an AVmultiplexing-resultant signal and a substitution playback audio signal;the AV multiplexing-resultant signal being generated by multiplexing 1)a compressively-encoding-resultant video signal which results fromcompressively encoding a video signal, 2) a firstcompressively-encoding-resultant audio signal which results fromcompressively encoding a first audio signal having a synchronizedrelation with the video signal, 3) a first reference clock informationsignal relating to synchronization between thecompressively-encoding-resultant video signal and the firstcompressively-encoding-resultant audio signal, and 4) an indicationtiming information signal designating an indication timing of the videosignal; the substitution playback audio signal being generated by 1)compressively encoding a second audio signal to get a secondcompressively-encoding-resultant audio signal, and 2) adding a secondreference clock information signal to the secondcompressively-encoding-resultant audio signal, wherein the second audiosignal can be reproduced instead of the first audio signal; wherein thesecond reference clock information signal added to the secondcompressively-encoding-resultant audio signal is equal to the firstreference clock information signal in the AV multiplexing-resultantsignal, and the second compressively-encoding-resultant audio signalcontains a multiplexed component being a sound-production-timinginformation signal based on one of the first and second reference clockinformation signals and relating to synchronization with the videosignal; and wherein the video signal is reproduced in response to thefirst reference clock information signal in the AVmultiplexing-resultant signal, and reproduction of the second audiosignal can be synchronized with the reproduction of the video signal inresponse to the reference clock information signal same as that used forthe reproduction of the video signal.

A twelfth aspect of this invention provides a computer program fortransmitting an information signal. The computer program comprises thesteps of generating a main AV multiplexing-resultant signal bymultiplexing 1) a compressively-encoding-resultant video signal whichresults from compressively encoding a video signal, 2) a firstcompressively-encoding-resultant audio signal which results fromcompressively encoding a first audio signal having a synchronizedrelation with the video signal, 3) a reference clock information signalrelating to synchronization between the compressively-encoding-resultantvideo signal and the first compressively-encoding-resultant audiosignal, and 4) an indication timing information signal designating anindication timing of the video signal on the basis of the referenceclock information signal; generating a substitution playback audiosignal by 1) compressively encoding a second audio signal to get asecond compressively-encoding-resultant audio signal, and 2) adding asound-production-timing information signal to the secondcompressively-encoding-resultant audio signal, thesound-production-timing information signal designating a soundproduction timing of the second compressively-encoding-resultant audiosignal on the basis of the reference clock information signal, whereinthe second audio signal can be reproduced instead of the first audiosignal; generating a substitution AV multiplexing-resultant signal as asignal for reproducing the compressively-encoding-resultant video signaltogether with the second compressively-encoding-resultant audio signal;feeding the substitution AV multiplexing-resultant signal to atransmission line; deriving the second compressively-encoding-resultantaudio signal from the substitution playback audio signal; and replacingthe first compressively-encoding-resultant audio signal in the main AVmultiplexing-resultant signal with the derived secondcompressively-encoding-resultant audio signal to change the main AVmultiplexing-resultant signal into the substitution AVmultiplexing-resultant signal before feeding the substitution AVmultiplexing-resultant signal to the transmission line.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an information-signal recording andreproducing system according to a first embodiment of this invention.

FIG. 2 is a block diagram of an MPEG encoder.

FIG. 3 is a block diagram of an MPEG decoder.

FIG. 4 is a block diagram of an MPEG-system decoding apparatus.

FIG. 5 is a diagram of the relation among an MPEG transport stream (TS),an MPEG program stream (PS), and packetized elementary streams (PESs).

FIG. 6 is a diagram of the format of a TS packet.

FIG. 7 is a flow diagram of the reception of TS packets in response topacket IDs (PIDs).

FIG. 8 is a block diagram of an information-signal recording apparatusin the first embodiment of this invention.

FIG. 9 is a block diagram of an information-signal reproducing apparatusin the first embodiment of this invention.

FIG. 10 is a diagram of the file structure of information recorded on aninformation-signal recording medium.

FIG. 11 is a diagram of the structure of a file “SIDE.ifo” in FIG. 10.

FIG. 12 is a diagram of the syntax structure of a second-level segment“GENERAL_IFO” in FIG. 11.

FIG. 13 is a diagram of the structure of a third-level segment“PR_IFO_0” in FIG. 11.

FIG. 14 is a diagram of the syntax structure of a fourth-level segment“PROG_IFO” in FIG. 13.

FIG. 15 is a diagram of the syntax structure of play list information“PLAYL_IFO” in FIG. 11.

FIG. 16 is a diagram of the syntax structure of index information“INDEX_IFO” in FIG. 13.

FIG. 17 is a diagram of the format of a transport stream (TS) recordedon a hard disk.

FIG. 18 is a block diagram of an information-signal transmissionapparatus in the first embodiment of this invention.

FIG. 19 is a flowchart of a segment of a control program for a CPU inFIG. 18.

DETAILED DESCRIPTION OF THE INVENTION First Embodiment

FIG. 1 shows an information-signal recording and reproducing systemaccording to a first embodiment of this invention. The system of FIG. 1includes an information-signal generating apparatus 10, anrecording-medium drive 20, an information-signal recording medium 25,and an information-signal reproducing apparatus 30.

The information-signal generating apparatus 10 has a vide encoder 11, anaudio encoder 11 a, and a multiplexer 15. The recording-medium drive 20is connected with the information-signal generating apparatus 10 and theinformation-signal reproducing apparatus 30. The recording-medium drive20 can write and read an information signal on and from the recordingmedium 25. A hard-disk writer/reader or an optical-disk writer/readermay be used as the recording-medium drive 20. A file containing aninformation signal can be recorded on the recording medium 25.Specifically, an AV (audio-visual) multiplexing-resultant file 25 a andan after-recorded audio file 25 b can be stored in the recording medium25. Preferably, the recording medium 25 has a recording surface on whichboth the AV multiplexing-resultant file 25 a and the after-recordedaudio file 25 b are recorded. The information-signal reproducingapparatus 30 has a demultiplexer 36, a signal selector 37, a videodecoder 45, and an audio decoder 45 a. Preferably, theinformation-signal reproducing apparatus 30 is connected to a monitor TVapparatus.

The system of FIG. 1 operates in a mode which can be selected fromdifferent ones including a recording mode, a first playback mode, and asecond playback mode. During the recording mode of operation, the videoencoder 11 receives a video signal accompanied with a first audiosignal. The video encoder 11 compressively encodes the received videosignal into a compressively-encoding-resultant video signal according tothe MPEG (Moving Picture Experts Group) standards for video encoding.Thus, the video encoder 11 includes an MPEG encoder (an MPEG videoencoder). The video encoder 11 feeds thecompressively-encoding-resultant video signal to a first input terminalof the multiplexer 15.

The audio encoder 11 a receives the first audio signal which accompaniesthe video signal. The audio encoder 11 a compressively encodes the firstaudio signal into a first compressively-encoding-resultant audio signalaccording to the MPEG standards for audio encoding. The audio encoder 11a feeds the first compressively-encoding-resultant audio signal to asecond input terminal of the multiplexer 15.

The device 15 multiplexes the compressively-encoding-resultant videosignal and the first compressively-encoding-resultant audio signal intoan AV multiplexing-resultant signal according to the MPEG systemstandards. The multiplexer 15 outputs the AV multiplexing-resultantsignal to the recording-medium drive 20. The recording-medium drive 20records the AV multiplexing-resultant signal on the recording medium 25as an AV multiplexing-resultant file 25 a.

After the recording of the AV multiplexing-resultant file 25 a iscompleted, a second audio signal starts to be recorded on the recordingmedium 25 as an after-recorded audio file 25 b separate from the AVmultiplexing-resultant file 25 a. Specifically, the audio encoder 11 areceives a second audio signal (an after-recording-purpose audio signalor a post-recording-purpose audio signal). The audio encoder 11 acompressively encodes the second audio signal into a secondcompressively-encoding-resultant audio signal according to the MPEGstandards for audio encoding. The audio encoder 11 a feeds the secondcompressively-encoding-resultant audio signal to the multiplexer 15. Themultiplexer 15 passes the second compressively-encoding-resultant audiosignal to the recording-medium drive 20. The recording-medium drive 20records the second compressively-encoding-resultant audio signal on therecording medium 25 as an after-recorded audio file 25 b.

The second audio signal differs in contents from the first audio signal.The second audio signal is generated in synchronism with thepreviously-mentioned video signal. For example, the second audio signalis generated while the video signal is reproduced. The first audiosignal is generated by picking up sound in a scene-shooting spotsimultaneously with shooting a scene to produce the video signal. On theother hand, the second audio signal is generated as a representation ofnarration or background music played after shooting the scene. Thesecond audio signal is also referred to as the after-recording-purposeaudio signal or the post-recording-purpose audio signal.

An AV signal having perfect audio and video information is referred toas a complete package signal. In the case where an AV program modifiedfrom that represented by a complete package signal is produced, by auser or a third person, from the video signal same as that in thecomplete package signal and an audio signal different from that in thecomplete package, the audio signal is defined as anafter-recording-purpose audio signal or a post-recording-purpose audiosignal.

The system of FIG. 1 is designed so that a second audio signal (anafter-recorded audio signal) and a video signal in a complete packagesignal can be synchronously reproduced, and the complete package signalcan also be reproduced. Furthermore, the system of FIG. 1 is designed toreduce the amount of information recorded on the recording medium 25.

As previously mentioned, the AV multiplexing-resultant signal isrecorded on the recording medium 25 as an AV multiplexing-resultant file25 a. In general, the AV multiplexing-resultant signal relates to acomplete package signal. Audio information and video information in theAV multiplexing-resultant file 25 a can be synchronously played back.The second audio signal is recorded on the recording medium 25 as anafter-recorded audio file 25 b separate from the AVmultiplexing-resultant file 25 a. As will be explained later, syncsignals are added to the AV multiplexing-resultant file 25 a and theafter-recorded audio file 25 b to make possible the synchronous playbackof audio information in the after-recorded audio file 25 b and videoinformation in the AV multiplexing-resultant file 25 a.

The modes of operation of the system in FIG. 1 include the firstplayback mode and the second playback mode. The first playback mode ofoperation is designed to reproduce a complete package signal. During thefirst playback mode of operation, the recording-medium drive 20 accessesan AV multiplexing-resultant file 25 a on an information-signalrecording medium 25 and reads out an AV multiplexing-resultant signaltherefrom. The recording-medium drive 20 feeds the read-out AVmultiplexing-resultant signal to the demultiplexer 36.

The demultiplexer 36 separates the AV multiplexing-resultant signal intoa compressively-encoding-resultant video signal and a firstcompressively-encoding-resultant audio signal according to the MPEGsystem standards. The demultiplexer 36 feeds thecompressively-encoding-resultant video signal to the video decoder 45.The demultiplexer 36 feeds the first compressively-encoding-resultantaudio signal to the signal selector 37.

The video decoder 45 expansively decodes thecompressively-encoding-resultant video signal into an original videosignal (a reproduced video signal) according to the MPEG standards forvideo decoding. Thus, the video decoder 45 includes an MPEG decoder (anMPEG video decoder). The video decoder 45 feeds the reproduced videosignal to a monitor TV apparatus. A display of the monitor TV apparatusindicates the reproduced video signal.

The signal selector 37 passes the first compressively-encoding-resultantaudio signal to the audio decoder 45 a. The audio decoder 45 aexpansively decodes the first compressively-encoding-resultant audiosignal into a first original audio signal (a first reproduced audiosignal) according to the MPEG standards for audio decoding. The audiodecoder 45 a feeds the first reproduced audio signal to the monitor TVapparatus. Loudspeakers of the monitor TV apparatus convert the firstreproduced audio signal into corresponding sound. The playback of thefirst reproduced audio signal is synchronous with the playback of thereproduced video signal.

The second playback mode of operation is designed to synchronouslyreproduce a second audio signal (an after-recorded audio signal) and avideo signal in a complete package signal. During the second playbackmode of operation, the recording-medium drive 20 alternately accesses anAV multiplexing-resultant file 25 a and an after-recorded audio file 25b on an information-signal recording medium 25 on a time sharing basis.The recording-medium drive 20 reads out an AV multiplexing-resultantsignal from the AV multiplexing-resultant file 25 a. Therecording-medium drive 20 feeds the read-out AV multiplexing-resultantsignal to the demultiplexer 36. The recording-medium drive 20 reads outa second compressively-encoding-resultant audio signal from theafter-recorded audio file 25 b. The recording-medium drive 20 feeds thesecond compressively-encoding-resultant audio signal to the signalselector 37.

The demultiplexer 36 separates the AV multiplexing-resultant signal intoa compressively-encoding-resultant video signal and a firstcompressively-encoding-resultant audio signal according to the MPEGsystem standards. The demultiplexer 36 feeds thecompressively-encoding-resultant video signal to the video decoder 45.The demultiplexer 36 feeds the first compressively-encoding-resultantaudio signal to the signal selector 37.

The video decoder 45 expansively decodes thecompressively-encoding-resultant video signal into an original videosignal (a reproduced video signal) according to the MPEG standards forvideo decoding. The video decoder 45 feeds the reproduced video signalto a monitor TV apparatus. A display of the monitor TV apparatusindicates the reproduced video signal.

The signal selector 37 selects the secondcompressively-encoding-resultant audio signal, and passes the selectedsignal to the audio decoder 45 a. Thus, the signal selector 37 rejectsthe first compressively-encoding-resultant audio signal coming from thedemultiplexer 36. The audio decoder 45 a expansively decodes the secondcompressively-encoding-resultant audio signal into a second originalaudio signal (a second reproduced audio signal) according to the MPEGstandards for audio decoding. The second reproduced audio signal is areproduced after-recording-purpose audio signal. The audio decoder 45 afeeds the second reproduced audio signal to the monitor IV apparatus.Loudspeakers of the monitor TV apparatus convert the second reproducedaudio signal (the reproduced after-recording-purpose audio signal) intocorresponding sound. The playback of the second reproduced audio signalis synchronous with the playback of the reproduced video signal. As willbe described later, the synchronization between the playback of thesecond reproduced audio signal and the playback of the reproduced videosignal is provided by sync signals added to the video information in theAV multiplexing-resultant file 25 a and the audio information in theafter-recorded audio file 25 b.

The system of FIG. 1 includes an user interface. The user can designatethe mode of operation of the system in FIG. 1 by actuating the userinterface. The signal selector 37 responds to a signal from the userinterface which represents the designated mode of operation is the firstplayback mode or the second playback mode. When the designated mode ofoperation is the first playback mode, the signal selector 37 passes thefirst compressively-encoding-resultant audio signal from thedemultiplexer 36 to the audio decoder 45 a. On the other hand, when thedesignated mode of operation is the second playback mode, the signalselector 37 selects the second compressively-encoding-resultant audiosignal and passes the selected signal to the audio decoder 45 a.

FIG. 2 shows an MPEG encoder 50 which can be used as that in the videoencoder 11 (see FIG. 1). The MPEG encoder 50 includes an adder 52operated as a subtracter. The subtracter 52 receives an input videosignal Vin via an input terminal 51. The subtracter 52 calculates aninter-frame prediction error, that is, a residual (an error) between apicture represented by the input video signal Vin and a final referencepicture (a final predicted picture) represented by output data from amotion-compensated predictor 65. The subtracter 52 outputs dataindicating the calculated residual (the calculated inter-frameprediction error).

Prediction implemented by the motion-compensated predictor 65 isdesigned as follows. Prediction directions are of three modes, that is,a direction from the past (“forward”), a direction from the future(“backward”), and a direction from both the past and future(“interpolative”). Accordingly, there are prediction from the past(“forward”), prediction from the future (“backward”), and predictionfrom both the past and future (“interpolative”). An actually-usedprediction direction can be changed MB by MB, where MB denotes amacro-block having 16 by 16 pixels. Every picture is divided into aprescribed number of MBs. The actually-used prediction direction isbasically decided by the type of every picture represented by the inputvideo signal Vin. Pictures are classified into P pictures (predictivecoded pictures), B pictures (bidirectionally predictive coded pictures),and I pictures (intra-coded pictures). MBs representative of a P pictureare of first and second modes. P-picture MBs of the first mode areencoded according to prediction from the past. P-picture MBs of thesecond mode are independently encoded without prediction. MBsrepresentative of a B picture are of first, second, third, and fourthmodes. B-picture MBs of the first mode are encoded according toprediction from the future. B-picture MBs of the second mode are encodedaccording to prediction from the past. B-picture MBs of the third modeare encoded according to prediction from both the past and future.B-picture MBs of the fourth mode are independently encoded withoutprediction. MBs representative of an I picture are independently encodedwithout prediction.

Motion compensation implemented by the motion-compensated predictor 65is designed as follows. According to motion compensation, patternmatching between two pictures is performed for each MB to detect amotion vector or motion vectors at an accuracy corresponding to a halfpel (a half pixel). One of the two pictures is given by the input videosignal Vin, and the other is referred to as a basic reference picture ora source picture for motion-compensated prediction. The basic referencepicture is shifted in accordance with detected motion vectors. A finalreference picture (a final predicted picture) is generated on the basisof the shift of the basic reference picture. In the absence of motioncompensation, the basic reference picture is directly used as a finalreference picture. Generally, a motion vector has a horizontal-directioncomponent and a vertical-direction component. Information representing adetected motion vector or detected motion vectors, and MC-mode(motion-compensation mode) information representing a predictiondirection or indicating a source picture from which prediction isimplemented are transmitted as added information relating to each MB. AGOP (group of pictures) is defined as a sequence of pictures startingfrom an 1 picture and ending at a picture immediately preceding a next 1picture. Generally, one GOP has about 15 successive pictures.

The residual-indicating data outputted from the subtracter 52 aresubjected to discrete cosine transform (DCT) by a DCT device 53.Specifically, the DCT device 53 divides every MB represented by theresidual-indicating data into 4 DCT blocks each having 8 by 8 pixels.The DCT device 53 subjects each DCT block to two-dimensional DCT togenerate data representing DCT coefficients. The DCT device 53 outputsthe generated DCT-coefficient data. In general, since a video signal isrich in low-frequency components, there occur a lot of effective DCTcoefficients corresponding to low frequencies.

DCT coefficients represented by the output data from the DCT device 53are quantized by a quantizer 54 in response to a quantization value. Thequantization value is equal to a quantization matrix multiplied by ascalar quantization scale. The quantization matrix has 8 by 8 elementsresulting from a process of weighting two-dimensional frequencycomponents according to visual sensation. The quantization includes astep of dividing DCT coefficients by the quantization value.

Quantization-resultant data outputted from the quantizer 54 are encodedby a VLC device 55 into data of a variable length code (VLC).Specifically, a direct-current (DC) component of thequantization-resultant data is encoded by DPCM (differential pulse codemodulation). Alternating-current (AC) components of thequantization-resultant data are scanned in zigzag along a direction froma high frequency toward a low frequency, and are subjected to Huffmanencoding in which data pieces having higher occurrence possibilities areassigned to shorter code words. The VLC device 55 receives themotion-vector information and the MC-mode information from themotion-compensated predictor 65. The VLC device 65 adds themotion-vector information and the MC-mode information to the VLC data.The resultant VLC data are temporarily stored in a buffer 56. The VLCdata are outputted from the buffer 56 at a prescribed transfer rate asencoding-resultant MPEG data in the form of a bit stream.

The buffer 56 informs a code amount controller 57 of the actual amountof encoding-resultant data for every MB, that is, the total number ofbits composing encoding-resultant data for every MB. The code amountcontroller 57 calculates an error between the actual amount ofencoding-resultant data and a target amount thereof. The code amountcontroller 57 adjusts the quantization scale used by the quantizer 54 inresponse to the calculated data amount error, and thereby controls theactual amount of encoding-resultant data.

The quantization-resultant data outputted from the quantizer 54 aresubjected to inverse quantization by an inverse quantizer 61, beingconverted back to DCT-coefficient data. The inverse quantizer 61 outputsthe DCT-coefficient data. The DCT-coefficient data outputted from theinverse quantizer 61 are subjected to inverse DCT by an inverse DCTdevice 62, being converted back into residual-indicating data. Theinverse DCT device 62 outputs the residual-indicating data. A residualpicture represented by the output data from the inverse DCT device 62and a final reference picture (a final predicted picture) represented bythe output data from the motion-compensated predictor 65 are added by anadder 63. The adder 63 outputs addition-resultant video data. Theaddition-resultant video data are temporarily stored in a memory 64.Video data are fed from the memory 64 to the motion-compensatedpredictor 65. The output video data from the memory 64 are used by themotion-compensated predictor 65 as an indication of a basic referencepicture or a source picture for motion-compensated prediction.

FIG. 3 shows an MPEG decoder 70 which can be used as that in the videodecoder 45 (see FIG. 1). The MPEG decoder 70 includes a buffer 72. Thebuffer 72 receives, via an input terminal 71, encoding-resultant MPEGdata in the form of a bit stream. The encoding-resultant MPEG data aretemporarily stored in the buffer 72 before being outputted to a VLD(variable length decoding) device 73 therefrom. The VLD device 73subjects the output data from the buffer 72 to VLD inverse with respectto VLC, thereby recovering DC-component data and AC-component data. TheAC-component data are scanned in zigzag along a direction from a highfrequency toward a low frequency. The DC-component data and theresultant AC-component data are arranged in a matrix having 8 by 8elements. The VLD device 73 outputs the matrix-arranged data to aninverse quantizer 74. The VLD device 73 extracts motion-vectorinformation and MC-mode information from the data outputted by thebuffer 72. The VLD device 73 outputs the motion-vector information andthe MC-mode information to a motion-compensated predictor 78.

The inverse quantizer 74 subjects the output data from the VLD device 73to inverse quantization responsive to the quantization value, therebyrecovering DCT-coefficient data. The inverse quantization includes astep of multiplying processed data values by the quantization value. Itshould be noted that the quantization value is equal to the quantizationmatrix multiplied by the scalar quantization scale. The DCT-coefficientdata are outputted from the inverse quantizer 74 to an inverse DCTdevice 75, being subjected to inverse DCT and hence being converted backinto residual-indicating data. The inverse DCT device 75 outputs theresidual-indicating data. A residual picture represented by the outputdata from the inverse DCT device 75 and a final reference picture (afinal predicted picture) represented by output data from themotion-compensated predictor 78 are added by an adder 76. The adder 76outputs addition-resultant video data as decoding-resultant video data(original video data). The decoding-resultant video data are transmittedtoward an external. The addition-resultant video data outputted from theadder 76 are temporarily stored in a memory 77. Video data are fed fromthe memory 77 to the motion-compensated predictor 78. The output videodata from the memory 77 are used by the motion-compensated predictor 78as an indication of a basic reference picture or a source picture formotion-compensated prediction. The motion-compensated predictor 78generates a final reference picture (a final predicted picture) inresponse to the basic reference picture, the motion-vector information,and the MC-mode information. The motion-compensated predictor 78 outputsdata representing the final reference picture.

The MPEG system standards prescribe the following five items.

1) Synchronous reproduction of a plurality of encoding-resultant bitstreams;2) Multiplexing a plurality of encoding-resultant bit streams into asingle bit stream;3) Initialization of encoding-resultant data in a buffer (the buffer 72in FIG. 3) at start of reproduction;4) Management of data in a buffer or buffers to get a continuousdecoding-resultant video signal and a continuous decoding-resultantaudio signal; and5) Management relating to the setting of timings such as a decodingstart timing and a playback indication timing.

Thus, according to the MPEG system standards, the managements relatingto encoding-resultant data are implemented. The encoding-resultant dataare formatted or processed into data of a packet structure, and aremultiplexed. Management data for implementing the operation steps in theabove items are added to the multiplexing-resultant data.

The multiplexing of information which conforms to the MPEG systemstandards includes a step of packeting the information. For example, inthe case where video information and audio information are required tobe multiplexed, each of the video information and the audio informationis divided into packets having a suitable length. Additional informationsuch as header information is added to each packet. Video-informationpackets and audio-information packets are mixed into a packet sequencebefore the packet sequence is transmitted. Therefore, thevideo-information packets and the audio-information packets aretransmitted on a time sharing basis. The header information containsinformation for identifying whether a related packet is of video, audio,or other, and also timing information for synchronization. In general,the packet length depends on a transmission medium and an application.For example, the packet length is equal to 53 bytes in the case of ATM(asynchronous transfer mode). The packet length is equal to 4 kilobytesin the case of an optical disc. According to the MPEG system standards,the packet length is variable, and can be set to an arbitrary value.

Data to be transmitted are divided into packs, and are packeted. Onepack is composed of several packets. A header portion of each pack isloaded with a pack-start-code and an SCR (system clock reference). Aheader of each packet is loaded with a stream ID and a time stamp. Thetime stamp contains timing information for synchronization between audioand video. The time stamp is of two types referred to as a DTS (decodingtime stamp) and a PTS (presentation time stamp) respectively. Thetransmitted data contain a periodically-occurring PCR (program clockreference) indicating a frequency of 27 MHz. A reference clock common todecoders can be locked to the frequency indicated by the PCR. The DTSdenotes the desired decoding start time for a first access unit in arelated packet, while the PTS denotes the desired indication start time(the desired playback start time) therefor. One access unit correspondsto one picture in the case of video. One access unit corresponds to 1152samples in the case of audio. The DTS and the PTS are given at a timeaccuracy based on the frequency indicated by the PCR.

The PCR is used as a reference clock information signal. In the case ofa video packet, the PTS is used as an indication timing informationsignal designating a timing of indication of related video information.In the case of an audio packet, the PTS is used as asound-production-timing information signal designating a timing ofconversion of related audio information into corresponding sound.

FIG. 4 shows an MPEG-system decoding apparatus which includes a systemdecoder 81 receiving multiplexing-resultant data (a system bit stream)containing video information and audio information. The system decoder81 demultiplexes the received data into video data and audio data. Thesystem decoder 81 outputs the audio data to an audio decoder (an MPEGaudio decoder) 82. The system decoder 81 outputs the video data to avideo decoder (an MPEG video decoder) 83. In addition, the systemdecoder 81 extracts every PCR, every video DTS, every audio DTS, everyvideo PTS, and every audio PTS from the received data. The systemdecoder 81 may extract every SCR from the received data. The systemdecoder 81 outputs the PCR (or the SCR) to a clock control circuit 84.The system decoder 81 outputs the video DTS, the audio DTS, the videoPTS, and the audio PTS to a comparison circuit 85.

The clock control circuit 84 includes an oscillator for generating areference clock signal denoting reference clock time. The clock controlcircuit 84 locks the frequency of oscillation of the oscillator to afrequency indicated by the PCR. Therefore, the reference clock time isdecided on the basis of the PCR. The clock control circuit 84 informsthe comparison circuit 85 of reference clock time, that is, time denotedby the reference clock signal.

The comparison circuit 85 compares time denoted by the audio DTS withthe reference clock time to decide whether or not the audio DTS timeagrees with the reference clock time. When the audio DTS time agreeswith the reference clock time, the comparison circuit 85 orders theaudio decoder 82 to start the decoding of the audio data. The audiodecoder 82 implements the decoding of the audio data. The audio decoder82 stores the decoding-resultant audio data into a memory 86. Thecomparison circuit 85 compares time denoted by the video DTS with thereference clock time to decide whether or not the video DTS time agreeswith the reference clock time. When the video DTS time agrees with thereference clock time, the comparison circuit 85 orders the video decoder83 to start the decoding of the video data. The video decoder 83implements the decoding of the video data. The video decoder 83 storesthe decoding-resultant video data into a memory 87.

The comparison circuit 85 compares time denoted by the audio PTS withthe reference clock time to decide whether or not the audio PTS timeagrees with the reference clock time. When the audio PTS time agreeswith the reference clock time, the comparison circuit 85 enables thememory 86 to output the decoding-resultant audio data for a playbackpurpose or a sound production purpose. The comparison circuit 85compares time denoted by the video PTS with the reference clock time todecide whether or not the video PTS time agrees with the reference clocktime. When the video PTS time agrees with the reference clock time, thecomparison circuit 85 enables the memory 87 to output thedecoding-resultant video data for an indication purpose. The video PTSand the audio PTS are preset so that the decoding-resultant video dataand the decoding-resultant audio data can be synchronously outputtedfrom the memories 86 and 87. Therefore, video information and audioinformation can be synchronously played back.

It is considered that a portion of the MPEG-system decoding apparatus ofFIG. 4 corresponds to a virtual decoder for temporarily storingdecoding-resultant data of plural types, and implementing synchronousreproduction of the plural-type data. The virtual decoder is referred toas an STD (system target decoder). A multiplexing-resultant bit streamis designed so that memories in the STD will neither overflow norunderflow.

The MPEG system standards prescribe a transport stream (TS) and aprogram stream (PS). The TS or the PS is formed by packetized elementarystreams (PESs) and packets loaded with other information. The PESs aredefined as intermediate streams for conversion or transformation betweena TS and a PS. The PESs are generated by packeting, for example,encoding-resultant MPEG video data, encoding-resultant MPEG audio data,or a private stream.

Video and audio data of content programs having common reference timecan be multiplexed into a PS. The PS includes a sequence of packets. Apacket layer of the PS is called “PES”. With reference to FIG. 5, thepacket layer of the PS is common to that of a TS so that compatibilitybetween the PS and the TS is provided. According to an STD model for aPS, the decoding mode for a bit stream to be decoded is switched to a PSside by a stream ID in a PES packet.

Video and audio data of content programs having common reference timecan also be multiplexed into a TS. Furthermore, video and audio data ofcontent programs different in reference time can be multiplexed into aTS. The TS is formed by a sequence of fixed-length packets, that is,188-byte packets (TS packets). The TS is designed for use in a systemwhere a data error or data errors are caused by a transmission line. Asunderstood from the above explanation, the TS is a stream representingmultiple content-programs. Although a TS packet ranks higher than a PESpacket, the TS packet is normally shorter than the PES packet.Generally, one PES packet is divided into segments, and the PES packetsegments are placed in plural TS packets respectively. According to anSTD model for a TS, the decoding mode for a bit stream to be decoded isswitched to a PS side by a packet ID (PID) in a TS packet.

With reference to FIG. 6, a TS packet has a header, an adaptation field(AF), and a payload which are sequentially arranged in that order. Theheader includes a sync byte, an error indicator, a unit start indicator,and a transport packet priority flag which are sequentially arranged inthat order. In the header, the transport packet priority flag isfollowed by a PID which indicates the type of the payload in the relatedpacket (the type of data in the payload in the related packet). In theheader, the PID is successively followed by a scramble controlinformation piece, an AF control information piece, and a cycliccounter. The tail of the header is occupied by the cyclic counter. TheAF control information piece represents whether the adaptation filed(AF) is present in or absent from the related packet. When the AFcontrol information piece represents the absence of the adaptationfield, the header is immediately followed by the payload. On the otherhand, when the AF control information piece represents the presence ofthe adaptation field, the header is successively followed by theadaptation field (AF) and the payload. The cyclic counter indicates thecontinuity about the related packet. In general, adaptation informationis placed in the adaptation field (AF) while content data such as videodata or audio data are placed in the payload. Ineffective data (dummydata) can be placed in the payload.

As shown in FIG. 6, the adaptation field includes an optional field. A48-bit front end of the optional field is loaded with a PCR.

An MPEG-system decoding apparatus handling a TS is designed to provide amechanism for detecting the PIDs in TS packets and classifying the TSpackets according to the detected PIDs during reception and decodingprocedures. Specifically, with reference to FIG. 7, TS packets aresearched for one having a PID=0 at a stage S1. In other words, a TSpacket having a PID=0 is received. The TS packet having a PID=0 isreferred to as a PAT (program association table) packet. The PAT packetis loaded with PAT information representing the relation between programnumbers (content-program ID numbers) PR and special PIDs on a linkbasis, that is, the relation between content programs and the specialPIDs. At a stage S2 following the stage S1, the PAT information in thePAT packet is detected. Each of the special PIDs corresponds to a PMT(program map table) packet. The PMT packet is loaded with PMTinformation representing the relation among the related content program,PIDs in video packets representing the related content program, and PIDsin audio packets representing the related content program. One isselected in advance from content programs as a desired content program.At the stage S2, one of the special PIDs which corresponds to thedesired content program is detected by referring to the PAT informationin the PAT packet. At a stage S3 following the stage S2, a PMT packethaving a PID identical with the detected special PID is received orsearched for. At a stage S4 subsequent to the stage S3, the received PMTpacket is accessed to get PMT information. According to the PMTinformation in the accessed PMT packet, PIDs in video packets and audiopackets representing the desired content program are detected. Inaddition, PIDs in sync clock packets relating to the desired contentprogram are detected. At a stage S5 a following the stage S4, TS packetscorresponding to the detected video packet PIDs are received. Videoinformation is extracted from the received TS packets. At a stage S5 bfollowing the stage S4, TS packets corresponding to the detected audiopacket PIDs are received. Audio information is extracted from thereceived TS packets. At a stage S5 c following the stage S4, TS packetscorresponding to the detected sync clock packet PIDs are received. Syncclock information is extracted from the received TS packets. In thisway, the video packets and the audio packets representing the desiredcontent program are accessed in response to the detected PIDs. Entryinto the desired content program is implemented by referring to the PATand the PMT. The PAT and the PMT are called PSI (program specificinformation).

FIG. 8 shows an information-signal generating apparatus 10 a which canbe used as the information-signal generating apparatus 10 in FIG. 1. Asshown in FIG. 8, the information-signal generating apparatus 10 aincludes a user interface 17 a connected with a CPU 17. The userinterface 17 a can be handled by a user. The CPU 17 has a combination ofan input/output port, a processing section, a ROM, and a RAM. The CPU 17operates in accordance with a control program stored in the ROM or theRAM. The control program is designed to enable the CPU 17 to implementoperation steps mentioned later. By handling the user interface 17 a,operation of the information-signal generating apparatus 10 a can bechanged among different modes including a first mode and a second mode.The first operation mode is designed to record a complete packagesignal, that is, main data containing both audio information and videoinformation. The second operation mode is designed to recordafter-recording-purpose audio data. When the user interface 17 a ishandled, an operation-mode designation signal is inputted therefrom intothe CPU 17. The operation-mode designation signal indicates which of thefirst operation mode and the second operation mode is desired. The CPU17 transfers the operation-mode designation signal to a signal selector16.

When the operation-mode designation signal indicates that the firstoperation mode is desired, that is, when the recording of main data isdesired, the information-signal generating apparatus 10 a operates asfollows. Input video data (a video signal) are fed to a video encoder11, and input audio data (a first audio signal) accompanying the inputvideo data are fed to an audio encoder 11 a. The video encoder 11 andthe audio encoder 11 a are similar to those in FIG. 1. In general, theinput video data and the input audio data represent a common contentprogram (a common audio-visual program), and are synchronous with eachother. The video encoder 11 implements the MPEG encoding of the inputvideo data to generate encoding-resultant video data. Theencoding-resultant video data are sent from the video encoder 11 to aPES packeting device 12. The audio encoder 11 a implements the MPEGencoding of the input audio data to generate encoding-resultant audiodata. The encoding-resultant audio data are sent from the audio encoder11 a to a PES packeting device 12 a. The PES packeting device 12converts the encoding-resultant video data into a sequence of PESpackets. The PES packet sequence is sent from the PES packeting device12 to a time stamp recorder 13. The PES packeting device 12 a convertsthe encoding-resultant audio data into a sequence of PES packets. ThePES packet sequence is sent from the PES packeting device 12 a to a timestamp recorder 13 a.

A signal generator 14 outputs a 27-MHz clock signal to the time stamprecorders 13 and 13 a. The time stamp recorder 13 generates timinginformation pieces, that is, a PCR and periodically-updated time stamps(a video PTS and a video DTS), in response to the 27-MHz clock signal.The time stamp recorder 13 records the PCR, the PTS, and the DTS in eachPES packet. Timing-information-added PES packets are sequentially sentfrom the time stamp recorder 13 to a multiplexer 15. The time stamprecorder 13 a generates a PCR and periodically-updated time stamps (anaudio PTS and an audio DTS) in response to the 27-MHz clock signal. Thetime stamp recorder 13 a records the PCR, the PTS, and the DTS in eachPES packet. Timing-information-added PES packets are sequentially sentfrom the time stamp recorder 13 a to the multiplexer 15. The multiplexer15 multiplexes the PES packets from the time stamp recorder 13 and thePES packets from the time stamp recorder 13 a to generatemultiplexing-resultant data (main data) of a PS form or a TS form. Themultiplexing-resultant data are sent from the multiplexer 15 to thesignal selector 16.

The signal selector 16 selects the multiplexing-resultant data (the maindata) in response to the operation-mode designation signal, and passesthe multiplexing-resultant data to a buffer 18. Themultiplexing-resultant data are stored in the buffer 18 before beingoutputted therefrom to a recording controller 19. The recordingcontroller 19 records the main data (the multiplexing-resultant data) onan information-signal recording medium 25 as an AVmultiplexing-resultant file 25 a having a name “PR . . . .dat” (see FIG.10).

The video PTS and the video DTS recorded by the time stamp recorder 13,and the audio PTS and the audio DTS recorded by the time stamp recorder13 a are in a relation such that the video information and the audioinformation can be synchronously reproduced from the recorded main data.

When the operation-mode designation signal indicates that the secondoperation mode is desired, that is, when the recording ofafter-recording-purpose audio data is desired, the information-signalgenerating apparatus 10 a operates as follows. The CPU 17 sendsidentification information (ID information) of after-recording-purposeaudio data to the PES packeting device 12 a. After-recording-purposeaudio data (a second audio signal) are fed to the audio encoder 11 a.The audio encoder 11 a implements the MPEG encoding of theafter-recording-purpose audio data to generate encoding-resultant audiodata. The encoding-resultant audio data are sent from the audio encoder11 a to the PES packeting device 12 a.

The PES packeting device 12 a adds the after-recording ID information tothe encoding-resultant audio data, and converts the ID-addedencoding-resultant video data into a sequence of PES packets. The PESpacket sequence is sent from the PES packeting device 12 a to the timestamp recorder 13 a.

An information-signal reproducing apparatus accesses the AVmultiplexing-resultant file 25 a on the recording medium 25, and readsout multiplexing-resultant data (main data) therefrom. Theinformation-signal reproducing apparatus separates and reproduces avideo signal from the read-out multiplexing-resultant data.Specifically, a reading controller 19 a possessed in common by theinformation-signal generating apparatus 10 a and the information-signalreproducing apparatus accesses the recording medium 25 and reproducesthe video signal. Furthermore, the reading controller 19 a extractsevery PCR, every video PTS, and every video DTS from the reproducedvideo signal. Alternatively, the PCR, the video PTS, and the video DTSrelating to the reproduced video signal may be obtained by anidentification information detector in the information-signalreproducing apparatus. The reading controller 19 a sends the extractedPCR to the signal generator 14. The reading controller 19 a informs theCPU 17 of the extracted video PTS and the extracted video DTS. Thesignal generator 14 locks the 27-MHz clock signal to the frequencyindicated by the extracted PCR.

The signal generator 14 outputs the 27-MHz clock signal to the timestamp recorder 13 a. The time stamp recorder 13 a generatestiming-information pieces, that is, a PCR and periodically-updated timestamps (an audio PTS and an audio DTS), in response to the 27-MHz clocksignal while being controlled by the CPU 17. The control of the timestamp recorder 13 a by the CPU 17 implements management of the audio PTSand the audio DTS in response to the extracted video PTS and theextracted video DTS, thereby providing a synchronized relation betweenthe audio information in the after-recording-purpose audio data and thevideo information in the recorded main data (the AVmultiplexing-resultant file 25 a). The time stamp recorder 13 a recordsthe PCR, the audio PTS, and the audio DTS in each PES packet.Preferably, the PCR, the audio PTS, and the audio DTS are set equal tothose which have been added to each audio PES packet during therecording of the main data. In this case, the timing relation of theafter-recorded audio data with the video information in the recordedmain data will be the same as that of the audio information in therecorded main data with the video information therein.Timing-information-added PES packets are sequentially sent from the timestamp recorder 13 a to the signal selector 16 without being propagatedthrough the multiplexer 15. The signal selector 16 selects the data (thePES packet sequence) outputted by the time stamp recorder 13 a inresponse to the operation-mode designation signal, and passes theselected data to a buffer 18 a. The selected data are stored in thebuffer 18 a before being outputted therefrom to the recording controller19. The recording controller 19 records the output data from the buffer18 a on the recording medium 25 as after-recorded audio data(post-recorded audio data). The after-recorded audio data on therecording medium 25 are in an after-recorded audio file 25 b having aname “AF- . . . .dat” (see FIG. 10). Preferably, the recording medium 25is alternately and periodically accessed by the recording controller 19and the information-signal reproducing apparatus (the reading controller19 a) on a time sharing basis.

Similarly, third and later audio signals (second and laterafter-recording-purpose audio signals) may be recorded on the recordingmedium 25 as second and later after-recorded audio files respectively.For example, an after-recorded audio file containing an M-thafter-recorded audio signal has a name “AF-M.dat”.

According to an information format mentioned later,after-recording-purpose audio data are recorded on an information-signalrecording medium 25 as play list information. Specifically, firstafter-recording-purpose audio data are recorded as a file having a name“AF-1.dat” in a PL folder (see FIG. 10). Also, second and laterafter-recording-purpose audio data are recorded. Furthermore, m-thafter-recording-purpose audio data are recorded as a file having a name“AF-m.dat” in the PL folder (see FIG. 10). Thus, “m” different types ofafter-recording-purpose audio data are recorded. An information piece“AF_number” (see FIG. 15) identifies after-recording-purpose audio data.Since the information piece “AF_number” has 8 bits (“0” is unused), upto 254 different types of after-recording-purpose audio data can behandled.

FIG. 9 shows an information-signal reproducing apparatus 30 a which canbe used as the information-signal reproducing apparatus 30 in FIG. 1. Asshown in FIG. 9, the information-signal reproducing apparatus 30 aincludes a user interface 32 connected with a CPU 33. The user interface32 can be handled by a user. The CPU 33 has a combination of aninput/output port, a processing section, a ROM, and a RAM. The CPU 33operates in accordance with a control program stored in the ROM or theRAM. The control program is designed to enable the CPU 33 to implementoperation steps mentioned later. By handling the user interface 32,operation of the information-signal reproducing-apparatus 30 a can bechanged among different modes including a first mode and a second mode.The first operation mode is designed to reproduce main data (an AVmultiplexing-resultant file 25 a). The second operation mode is designedto reproduce after-recorded audio data (an after-recorded audio file 25b) instead of audio information in main data while synchronouslyreproducing video information in the main data. When the user interface32 is handled, an operation-mode designation signal is inputtedtherefrom into the CPU 33. The operation-mode designation signalindicates which of the first operation mode and the second operationmode is desired. The CPU 33 transfers the operation-mode designationsignal to a signal selector 37. Also, identification (ID) informationcan be inputted into the CPU 33 by handling the user interface 32. TheCPU 33 transfers the identification information to an identificationinformation detector 34.

When the operation-mode designation signal indicates that the firstoperation mode is desired, that is, when the reproduction of main data(an AV multiplexing-resultant file 25 a) is desired, theinformation-signal reproducing apparatus 30 a operates as follows. Asignal for identifying desired main data is inputted into the CPU 33 byhandling the user interface 32. The identifying signal represents thedesignation number (the identification number) assigned to the desiredmain data. The CPU 33 transfers the identifying signal to theidentification information detector 34. The identification informationdetector 34 derives the identification number of the desired main datafrom the identifying signal. The identification information detector 34notifies a reading controller 31 of the identification number of thedesired main data. The identification information detector 34 orders thereading controller 31 to read out, from an information-signal recordingmedium 25, a main-data file (an AV multiplexing-resultant file 25 a)having a name “PR . . . .dat” corresponding to the identification numberof the desired main data. Thus, the reading controller 31 implements theread-out of the desired main data from the recording medium 25. In thisway, the reading controller 31 reads out desired main data, that is,desired multiplexing-resultant data, from the recording medium 25. Theread-out main data are sent from the reading controller 31 to a buffer35. The main data are stored in the buffer before being outputtedtherefrom to a demultiplexer 36. The demultiplexer 36 separates the maindata into video data and audio data. The video data are sent from thedemultiplexer 36 to a time stamp detector 43. The audio data are sentfrom the demultiplexer 36 to the signal selector 37. The signal selector37 selects the audio data from the demultiplexer 36 in response to theoperation-mode designation signal, and passes the selected audio data toa time stamp detector 43 a.

The time stamp detector 43 detects timing information (every PCR, everyvideo PTS, and every video DTS) in the video data. The time stampdetector 43 sends the detected PCR, the detected video PTS, and thedetected video DTS to a time stamp comparator 42. The time stampdetector 43 passes the video data to a PES de-packeting device 44. Thetime stamp detector 43 a detects timing information (every PCR, everyaudio PTS, and every audio DTS) in the audio data. The time stampdetector 43 a sends the detected PCR, the detected audio PTS, and thedetected audio DTS to the time stamp comparator 42. The time stampdetector 43 a passes the audio data to a PES de-packeting device 44 a.

The PES de-packeting device 44 de-packets the video data (a sequence ofPES packets) to generate de-packeting-resultant video data. The PESde-packeting device 44 outputs the de-packeting-resultant video data toa video decoder 45. The PES de-packeting device 44 a de-packets theaudio data (a sequence of PES packets) to generatede-packeting-resultant audio data. The PES de-packeting device 44 aoutputs the de-packeting-resultant audio data to an audio decoder 45 a.The video decoder 45 and the audio decoder 45 a are similar to those inFIG. 1. The video decoder 45 implements the MPEG decoding of the videodata to generate decoding-resultant video data. The video decoder 45stores the decoding-resultant video data into a memory 46. The audiodecoder 45 a implements the MPEG decoding of the audio data to generatedecoding-resultant audio data. The audio decoder 45 a stores thedecoding-resultant audio data into a memory 46 a.

A signal generator 41 outputs a 27-MHz clock signal to the time stampcomparator 42. On the basis of the 27-MHz clock signal, the time stampcomparator 42 generates a reference clock signal denoting referenceclock time. The time stamp comparator 42 locks the frequency of thereference clock signal to a frequency indicated by the PCR. Therefore,the reference clock time is decided by the PCR. The time stampcomparator 42 compares time denoted by the video DTS with the referenceclock time to decide whether or not the video DTS time agrees with thereference clock time. When the video DTS time agrees with the referenceclock time, the time stamp comparator 42 orders the video decoder 45 tostart the decoding of the video data originating from the PES packethaving the related video DTS. The time stamp comparator 42 compares timedenoted by the audio DTS with the reference clock time to decide whetheror not the audio DTS time agrees with the reference clock time. When theaudio DTS time agrees with the reference clock time, the time stampcomparator 42 orders the audio decoder 45 a to start the decoding of theaudio data originating from the PES packet having the related audio DTS.

The time stamp comparator 42 compares time denoted by the video PTS withthe reference clock time to decide whether or not the video PTS timeagrees with the reference clock time. When the video PTS time agreeswith the reference clock time, the time stamp comparator 42 orders thememory 46 to output the decoding-resultant video data to a displaymonitor 27 for an indication purpose. The time stamp comparator 42compares time denoted by the audio PTS with the reference clock time todecide whether or not the audio PTS time agrees with the reference clocktime. When the audio PTS time agrees with the reference clock time, thetime stamp comparator 42 orders the memory 46 a to output thedecoding-resultant audio data to loudspeakers 28 for a playback purposeor a sound production purpose. The video PTS and the audio PTS arepreset so that the decoding-resultant video data and thedecoding-resultant audio data can be synchronously played back.

When the operation-mode designation signal indicates that the secondoperation mode is desired, that is, when the reproduction ofafter-recorded audio data and main-data video information is desired,the information-signal reproducing apparatus 30 a operates as follows.Signals for identifying desired main data and desired after-recordedaudio data are inputted into the CPU 33 by handling the user interface32. The identifying signals correspond to information pieces “PR_number”and “AF_number” in FIG. 15 which represent the designation numbers (theidentification numbers) assigned to the desired main data and thedesired after-recorded audio data. The CPU 33 transfers the identifyingsignals to the identification information detector 34. The readingcontroller 31 reads out play list information “PLAYL_IFO” (see FIG. 15)from an information-signal recording medium 25, and sends the play listinformation “PLAYL_IFO” to the identification information detector 34.The identification information detector 34 detects the identificationnumbers of the desired main data and the desired after-recorded audiodata in response to the identifying signals by referring to the playlist information “PLAYL_IFO”. The identification information detector 34notifies the reading controller 31 of the identification numbers of thedesired main data and the desired after-recorded audio data. Theidentification information detector 34 orders the reading controller 31to alternately read out, from the recording medium 25, a main-data file(an AV multiplexing-resultant file 25 a) and an after-recorded audiofile 25 b having names corresponding to the identification numbers ofthe desired main data and the desired after-recorded audio data. Thus,the reading controller 31 implements the read-out of the desired maindata and the desired after-recorded audio data from the recording medium25 on an alternate time-sharing burst basis. The read-out main data aresent from the reading controller 31 to the buffer 35. The main data arestored in the buffer 35 before being outputted therefrom to thedemultiplexer 36. The demultiplexer 36 separates the main data intovideo data and audio data. The video data are sent from thedemultiplexer 36 to the time stamp detector 43. The audio data are sentfrom the demultiplexer 36 to the signal selector 37. The read-outafter-recorded audio data are sent from the reading controller 31 to thebuffer 35 a. The after-recorded audio data are stored in the buffer 35 abefore being outputted therefrom to the signal selector 37. The signalselector 37 selects the after-recorded audio data from the buffer 35 ain response to the operation-mode designation signal, and passes theselected after-recorded audio data to the time stamp detector 43 a. Inother words, the signal selector 37 rejects the audio data from thedemultiplexer 36.

The time stamp detector 43 detects timing information (every PCR, everyvideo PTS, and every video DTS) in the video data. The time stampdetector 43 sends the detected PCR, the detected video PTS, and thedetected video DTS to the time stamp comparator 42. The time stampdetector 43 passes the video data to the PES de-packeting device 44. Thetime stamp detector 43 a detects timing information (every PCR, everyaudio PTS, and every audio DTS) in the after-recorded audio data. Thetime stamp detector 43 a sends the detected PCR, the detected audio PTS,and the detected audio DTS to the time stamp comparator 42. The timestamp detector 43 a passes the after-recorded audio data to the PESde-packeting device 44 a.

The PES de-packeting device 44 de-packets the video data (a sequence ofPES packets) to generate de-packeting-resultant video data. The PESde-packeting device 44 outputs the de-packeting-resultant video data tothe video decoder 45. The PES de-packeting device 44 a de-packets theafter-recorded audio data (a sequence of PES packets) to generatede-packeting-resultant audio data. The PES de-packeting device 44 aoutputs the de-packeting-resultant audio data to the audio decoder 45 a.The video decoder 45 implements the MPEG decoding of the video data togenerate decoding-resultant video data. The video decoder 45 stores thedecoding-resultant video data into the memory 46. The audio decoder 45 aimplements the MPEG decoding of the audio data to generatedecoding-resultant audio data. The audio decoder 45 a stores thedecoding-resultant audio data into the memory 46 a.

The signal generator 41 outputs the 27-MHz clock signal to the timestamp comparator 42. On the basis of the 27-MHz clock signal, the timestamp comparator 42 generates a reference clock signal denotingreference clock time. The time stamp comparator 42 locks the frequencyof the reference clock signal to a frequency indicated by the PCR.Therefore, the reference clock time is decided by the PCR. The timestamp comparator 42 compares time denoted by the video DTS with thereference clock time to decide whether or not the video DTS time agreeswith the reference clock time. When the video DTS time agrees with thereference clock time, the time stamp comparator 42 orders the videodecoder 45 to start the decoding of the video data originating from thePES packet having the related video DTS. The time stamp comparator 42compares time denoted by the audio DTS with the reference clock time todecide whether or not the audio DTS time agrees with the reference clocktime. When the audio DTS time agrees with the reference clock time, thetime stamp comparator 42 orders the audio decoder 45 a to start thedecoding of the audio data originating from the PES packet having therelated audio DTS.

The time stamp comparator 42 compares time denoted by the video PTS withthe reference clock time to decide whether or not the video PTS timeagrees with the reference clock time. When the video PTS time agreeswith the reference clock time, the time stamp comparator 42 orders thememory 46 to output the decoding-resultant video data to the displaymonitor 27 for an indication purpose. The time stamp comparator 42compares time denoted by the audio PTS with the reference clock time todecide whether or not the audio PTS time agrees with the reference clocktime. When the audio PTS time agrees with the reference clock time, thetime stamp comparator 42 starts the memory 46 a to output thedecoding-resultant audio data to the loudspeakers 28 for a playbackpurpose or a sound production purpose. The video PTS and the audio PTSare preset so that the decoding-resultant video data and thedecoding-resultant audio data (the after-recorded audio data) can besynchronously played back.

The format of information recorded on an information-signal recordingmedium 25 will be explained below. As shown in FIG. 10, the recordingmedium 25 stores a ROOT directory under which a folder named “LIB(library)” is placed. Under the folder “LIB”, there are a plurality offiles named “SIDE.ifo” (“SIDE0.ifo”, “SIDE1.ifo”, . . . , and“SIDEk.ifo”). The files “SIDE.ifo” are loaded with side informationrelating to a plurality of content programs, that is, audio-visual (AV)programs.

In addition, under the folder “LIB”, there are folders “PR0”, “PR1”, . .. , and “PRn” loaded with information pieces “PR0.dat”, “PR1.dat”, and“PRn.dat”, respectively. The information pieces “PR0.dat”, “PR1.dat”,and “PRn.dat” are designed for link with AV multiplexing-resultant bitstreams.

Furthermore, under the folder “LIB”, there are folders “PL0”, “PL1”, . .. , and “PLn” for containing after-recorded audio files (files loadedwith after-recorded audio information). For example, in the case where“m” after-recorded audio files relating to the folder “PR0” are made andrecorded, list information pieces “AF0-1.dat”, “AF0-2.dat”, . . . ,“AF0-m.dat” are placed in the folder “PL0” as information for link withthe after-recorded audio files.

As understood from the above description, link information relating toAV multiplexing-resultant files and after-recorded audio files isrecorded on the recording medium 25. Desired content information can bereproduced from the recording medium 25 on the basis of the linkinformation.

With reference to FIG. 11, each file “SIDE.ifo” is in a format having ahierarchical structure. Specifically, each file “SIDE.ifo” has afirst-level segment “TOTAL_MANAGER_IFO” containing second-level segments“GENERAL_IFO” and “CNTNT_IFO”. The second-level segment “GENERAL_IFO” isloaded with parameters relating to the whole of the present sideinformation.

The second-level segment “GENERAL_IFO” is of a syntax structure shown inFIG. 12. Specifically, the second-level segment “GENERAL_IFO” includesinformation pieces having syntax names “System_id”, “TMG_IFO_length”,“Num of PR_IFO”, “Num of PL_IFO”, “Start Address of PR_IFO”, and “StartAddress of PL-IFO”, respectively. The information piece “System_id” is a32-bit signal representing the type of the present informationstructure. The information piece “TMG_IFO_length” is a 32-bit signalrepresenting the whole manager length. The information piece “Num ofPR_IFO” is an 8-bit signal representing the number of programinformation pieces “PR_IFO” which will be explained later. Theinformation piece “Num of PL_IFO” is an 8-bit signal representing thenumber of after-recorded information pieces “PL_IFO” which will beexplained later. The information piece “Start Address of PR_IFO” is a32-bit signal representing the head address of a first programinformation piece “PR_IFO_0”. The information piece “Start Address ofPL_IFO” is a 32-bit signal representing the head address of a firstafter-recorded information piece “PL_IFO_0”.

The second-level segment “CNTNT_IFO” in FIG. 11 contains third-levelsegments “PR_IFO_0”, “PR_IFO_1”, . . . , and “PR_IFO_m” loaded withinformation pieces which relate to content programs (or AVmultiplexing-resultant files) respectively.

Furthermore, the second-level segment “CNTNT_IFO” contains third-levelsegments “PL_IFO_0”, “PL_IFO_1”, . . . , and “PL_IFO_n” loaded withinformation pieces for after-recorded audio data which relate to thecontent programs (or the AV multiplexing-resultant files) respectively.For example, in the case where after-recorded audio data correspondingto the third-level segment “PR_IFO_0” are present, the third-levelsegment “PL-IFO_0” contains a fourth-level segment “PLAYL_IFO” loadedwith information (play list information) relating to the after-recordedaudio file.

The third-level segments “PR_IFO_0”, “PR_IFO_1”, . . . , and “PR_IFO_m”are similar in structure. Only the third-level segment “PR_IFO_0” willbe explained in more detail. As shown in FIG. 13, the third-levelsegment “PR_IFO_0” contains fourth-level segments “PROG_IFO” and“IDX_IFO”. The fourth-level segment “PROG_IFO” is loaded withcontent-program-related information. The fourth-level segment “IDX_IFO”contains fifth-level segments “IDX_IFO_0”, “IDX_IFO_1”, . . . , and“IDX_IFO_n” loaded with information pieces which relate to respectiveindexes of the related audio-visual program. The fifth-level segments“IDX_IFO_0”, “IDX_IFO_1”, . . . , and “IDX_IFO_n” are similar instructure. For example, the fifth-level segment “IDX_IFO_0” has asixth-level segment “INDEX_IFO” in which a portion of the relatedaudio-visual program can be registered as index information.

The fourth-level segment “PROG_IFO” is of a syntax structure shown inFIG. 14. Specifically, each fourth-level segment “PROG_IFO” includesinformation pieces having syntax names “Size of PROG_IFO”, “PR number”,“Content type”, and “Component type”, respectively. The informationpiece “Size of PROG_IFO” is a 32-bit signal representing the size of thepresent fourth-level segment “PROG_IFO”. The information piece “PRnumber” is an 8-bit signal representing the designation number (theidentification number) assigned to the related audio-visual program. ThePR number is variable among “0”-“255” corresponding to differentaudio-visual programs respectively. The information piece “Content type”is an 8-bit signal representing the type of the related audio-visualprogram. The information piece “Component type” is an 8-bit signalrepresenting the type of related data, that is, representing whetherrelated data are of video, audio, or other.

As previously mentioned, in the presence of an after-recorded audiofile, play list information “PLAYL_IFO” is provided. In the presence of“m” after-recorded audio files corresponding to the folder “PR0”, listinformation pieces “AF0-1.dat”, “AF0-2.dat”, . . . , “AF0-m.dat” areplaced in the folder “PL0” as information for link with theafter-recorded audio files.

The play list information “PLAYL_IFO” is of a syntax structure shown inFIG. 15. Specifically, the play list information “PLAYL_IFO” includesinformation pieces having syntax names “PR_number” and “AF_number”respectively. The information piece “PR_number” is an 8-bit signalrepresenting the designation number (the identification number) assignedto the related main data, that is, the related audio-visual program. Theinformation piece “AF_number” is an 8-bit signal representing thedesignation number (the identification number) assigned to the relatedafter-recorded audio data.

Regarding each of “n+1” AV multiplexing-resultant streams “PRj.dat (j=0,1, . . . , n)” in the folders “PRj” of FIG. 10, “m” after-recorded audiodata can be registered in accordance with user's request. When thenumber “m” is equal to “0”, an AV multiplexing-resultant file is usedwithout employing after-recorded audio data. When the number “m” isequal to “1” or greater, the recording of after-recording-purpose audiodata is permitted. In this case, at least one after-recorded audio fileis made and recorded. Also, the after-recorded audio data are reproducedor transmitted.

The index information “INDEX_IFO” in FIG. 13 is of a syntax structure inFIG. 16. Specifically, the index information “INDEX_IFO” includesinformation pieces having syntax names “INDEX number”, “Playback Time”,“Start Address”, and “End Address”, respectively. The information piece“INDEX number” is an 8-bit signal representing the serial numberassigned to the related index. The information piece “Playback Time” isa 40-bit signal representing the playback time of the related index. Theinformation piece “Start Address” is a 64-bit signal representing theaddress of the starting point of the related index. The informationpiece “End Address” is a 64-bit signal representing the address of theending point of the related index.

In this way, side information shown in FIG. 10 is designed. Recordinginformation of files relating to folders “PR” and “PL” and listinformation pieces “AF” are generated on the basis of the sideinformation. Prescribed AV and audio files are recorded on the recordingmedium 25 by referring to the recording information. Designated AV andaudio files are read out from the recording medium 25 by referring tothe recording information. Accordingly, a video signal accompanied witha desired audio signal can be played back while the desired audio signalis reproduced.

The recording medium 25 includes, for example, a DVD-RAM, a DVD-RW, aDVD-R, a magneto-optical disk, or a hard disk. As previously mentioned,during the recording of after-recording-purpose audio data on therecording medium 25, main data are read out from the recording medium25. The recording of the after-recording-purpose audio data and theread-out of the main data are alternately implemented on a time sharingbasis at periods each corresponding to a plurality of packets.Preferably, the after-recording-purpose audio data are recorded on asurface of the recording medium 25 where the main data are alsorecorded.

FIG. 17 shows the format of a TS recorded on a hard disk whichconstitutes the recording medium 25. As shown in FIG. 17, the TS isformed by a sequence of TS units. Each TS unit is composed of packets.Each packet is composed of a 25-bit time stamp and a 188-byte MPEG TSpacket. The MPEG TS packet is identical with a TS packet in FIG. 5. Inthe case where audio data in an AV multiplexing-resultant file 25 a arereplaced by audio data (after-recorded audio data) in an after-recordedaudio file 25 b, the synchronized relation between the after-recordedaudio data and the video data in the AV multiplexing-resultant file 25 acan be maintained since a sync-providing time stamp is added to each TSpacket.

FIG. 18 shows an information-signal transmission apparatus 30 b. Asshown in FIG. 18, the transmission apparatus 30 b includes a userinterface 32E connected with a CPU 33E. The user interface 32E can behandled by a user. The CPU 33E has a combination of an input/outputport, a processing section, a ROM, and a RAM. The CPU 33E operates inaccordance with a control program stored in the ROM or the RAM. Thecontrol program is designed to enable the CPU 33E to implement operationsteps mentioned later. By handling the user interface 32E, operation ofthe transmission apparatus 30 b can be changed among different modesincluding a first mode and a second mode. The first operation mode isdesigned to transmit main data (an AV multiplexing-resultant file 25 a).The second operation mode is designed to transmit after-recorded audiodata (an after-recorded audio file 25 b) instead of audio information inmain data while transmitting video information in the main data. Whenthe user interface 32E is handled, an operation-mode designation signalis inputted therefrom into the CPU 33E. An operation-mode designationsignal may be sent to the CPU 33E from an information-signal receptionapparatus which is located at a position remote from the transmissionapparatus 30 b. The operation-mode designation signal indicates which ofthe first operation mode and the second operation mode is desired. TheCPU 33E transfers the operation-mode designation signal to a signalselector 38 which can also be operated as a packet selector.Furthermore, identification (ID) information can be inputted into theCPU 33E by handling the user interface 32E. Identification informationmay be sent to the CPU 33E from the reception apparatus. The CPU 33Etransfers the identification information to an identificationinformation detector 34.

When the operation-mode designation signal indicates that the firstoperation mode is desired, that is, when the transmission of main data(an AV multiplexing-resultant file 25 a) is desired, the transmissionapparatus 30 b operates as follows. A reading controller 31 reads outmain data, that is, AV multiplexing-resultant data, from aninformation-signal recording medium 25. The read-out main data are sentfrom the reading controller 31 to a buffer 35. The main data are storedin the buffer 35 before being outputted therefrom to the signal selector38. The signal selector 38 selects the main data from the buffer 35 inresponse to the operation-mode designation signal, and passes the maindata to a buffer 39. The main data are stored in the buffer 39 beforebeing outputted therefrom to a transmission line. The main data (the AVmultiplexing-resultant data) are propagated to the reception apparatusalong the transmission line. The reception apparatus synchronously playsback audio information and video information in the main data.

When the operation-mode designation signal indicates that the secondoperation mode is desired, that is, when the transmission ofafter-recorded audio data and main-data video information is desired,the transmission apparatus 30 b operates as follows. Signals foridentifying desired main data and desired after-recorded audio data areinputted into the CPU 33E by handling the user interface 32E. Theidentifying signals may be sent to the CPU 33E from the receptionapparatus. The identifying signals correspond to information pieces“PR_number” and “AF_number” in FIG. 15 which represent the designationnumbers (the identification numbers) assigned to the desired main dataand the desired after-recorded audio data. The CPU 33E transfers theidentifying signals to the identification information detector 34. Thereading controller 31 reads out play list information “PLAYL_IFO” froman information-signal recording medium 25. The reading controller 31sends the play list information “PLAYL_IFO” to the identificationinformation detector 34. The identification information detector 34detects the identification numbers of the desired main data and thedesired after-recorded audio data in response to the identifying signalsby referring to the play list information “PIAYL_IFO” (see FIG. 15). Theidentification information detector 34 notifies the reading controller31 of the identification numbers of the desired main data and thedesired after-recorded audio data. The identification informationdetector 34 orders the reading controller 31 to alternately read out,from the recording medium 25, a main-data file (an AVmultiplexing-resultant file 25 a) and an after-recorded audio file 25 bhaving names corresponding to the identification numbers of the desiredmain data and the desired after-recorded audio data. Thus, the readingcontroller 31 implements the read-out of the desired main data and thedesired after-recorded audio data from the recording medium 25 on analternate time-sharing burst basis. The read-out main data are sent fromthe reading controller 31 to the buffer 35. The main data are stored inthe buffer 35 before being outputted therefrom to the signal selector38. The read-out after-recorded audio data are sent from the readingcontroller 31 to a buffer 35 a. The after-recorded audio data are storedin the buffer 35 a before being outputted therefrom to the signalselector 38. The signal selector (the packet selector) 38 replaces audiodata in the main data with the after-recorded audio data from the buffer35 a in response to the operation-mode designation signal, and therebyconverts the original main data into new main data (new AVmultiplexing-resultant data). The signal selector 38 implements theaudio-data replacement on a packet-by-packet basis (anelement-by-element basis). The new main data are sent from the signalselector 38 to the buffer 39. The new main data are stored in the buffer39 before being outputted therefrom to the transmission line. The newmain data (the new AV multiplexing-resultant data) are propagated to thereception apparatus along the transmission line. The reception apparatussynchronously plays back audio information and video information in themain data. During the conversion of the original main data into the newmain data, it is unnecessary to alter every PCR, every DTS, and everyPTS. Preferably, the signal selector 38 includes a calculator. Thecalculator may be formed by the CPU 33E. For packets of the new maindata which have been subjected to the audio-data replacement, thecalculator computes new CRC (cyclic redundancy check) code words andreplaces old CRC code words with the new ones.

FIG. 19 is a flowchart of a segment of the control program for the CPU33E. As shown in FIG. 19, a first step S11 controls the readingcontroller 31 to read out main data (an AV multiplexing-resultant file25 a) from an information-signal recording medium 25.

A step S12 following the step S11 decides whether or not audioinformation in the read-out main data is required to be replaced withafter-recorded audio information. When audio information in the read-outmain data is required to be replaced with after-recorded audioinformation, the program advances from the step S12 to a step S13.Otherwise, the program jumps from the step S12 to a step S17.

The step S13 controls the identification information detector 34 and thereading controller 31 to read out, from the recording medium 25, anafter-recorded audio file 25 b (after-recorded audio data) having a namecorresponding to a designated identification number.

A step S14 subsequent to the step S13 divides the read-out AVmultiplexing-resultant file 25 a and the read-out after-recorded audiofile 25 b, that is, the read-out main data and the read-outafter-recorded audio data, into packets (elements).

A step S15 following the step S14 replaces audio data in the read-outmain data with the read-out after-recorded audio data equal in playbacktiming to the former audio data on a packet-by-packet basis. Thereby,the step S15 converts the original main data into new main data (a newAV multiplexing-resultant file).

A step S16 subsequent to the step S15 computes new CRC (cyclicredundancy check) code words and replaces old CRC code words with thenew ones for packets of the new main data which have been subjected tothe audio-data replacement. Thus, the step S16 generatesCRC-code-revised main data (a CRC-code-revised AV multiplexing-resultantfile). After the step S16, the program advances to the step S17.

The step S17 transmits either the AV multiplexing-resultant file givenby the step S11 or the AV multiplexing-resultant file given by the stepS16 to a transmission line.

A step S18 following the step S17 decides whether or not thetransmission of the AV multiplexing-resultant file has been completed.When the transmission of the AV multiplexing-resultant file has not yetbeen completed, the program returns from the step S18 to the step S11.In this case, the step S11 and the later steps are repeated. On theother hand, when the transmission of the AV multiplexing-resultant filehas been completed, the program exits from the step S18 and then thecurrent execution of the program segment ends. The step S18 may alsorespond to a transmission-halt request. In this case, upon the receptionof a transmission-halt request, the program exits from the step S18 andthen the current execution of the program segment ends regardless ofwhether or not the transmission of the AV multiplexing-resultant filehas been completed.

As previously mentioned, AV multiplexing-resultant data which conform tothe MPEG system standards are recorded on or reproduced from aninformation-signal recording medium. The AV multiplexing-resultant datacontain audio information and video information accompanied with theaudio information. Furthermore, after-recording-purpose audio data whichconform to the MPEG system standards are recorded on and reproduced fromthe recording medium. The after-recording-purpose audio data have asynchronized relation with the video information in the AVmultiplexing-resultant data. The after-recording-purpose audio data andthe video information in the AV multiplexing-resultant data aresynchronously played back. The AV multiplexing-resultant data can betransmitted from the recording medium toward an information-signalreception apparatus via a transmission line. Also, theafter-recording-purpose audio data and the video information in the AVmultiplexing-resultant data which compose new AV multiplexing-resultantdata can be transmitted from the recording medium toward the receptionapparatus via the transmission line.

As previously mentioned, an audio signal and a video signal arecompressively encoded. The encoding-resultant data are divided into unitblocks (packets). A time stamp for audio-video synchronous playback isadded to each unit block. The time-stamp-added data are recorded on aninformation-signal recording medium.

A video signal and a first audio signal are compressively encoded andmultiplexed into main data, and the main data are recorded on aninformation-signal recording medium. A second audio signal, that is, anafter-recording-purpose audio signal, is compressively encoded intoafter-recording-purpose audio data. The after-recording-purpose audiodata are recorded on the recording medium without being multiplexed withthe video data in the main data.

Therefore, simple signal processing enables after-recording-purposeaudio data to be recorded on an information-signal recording medium. Inaddition, a plurality of after-recording-purpose audio signalsaccompanying same video information can be recorded on the recordingmedium. Each of the after-recording-purpose audio signals can be playedback synchronously with the video information. As previously mentioned,a first audio signal and a video signal are multiplexed before beingrecorded on an information-signal recording medium as an AVmultiplexing-resultant file. During playback, the first audio signal inthe AV multiplexing-resultant file can be replaced with each ofafter-recording-purpose audio signals.

Timing information added to after-recording-purpose audio data iscomposed of a PCR, a DTS, and a PTS generated on the basis of main data.It should be noted that a PCR, a DTS, and a PTS forafter-recording-purpose audio data may be generated independently ofmain data.

Information for getting a reference timing may be placed in a header oftiming information in after-recording-purpose audio data. In this case,the playback of the after-recording-purpose audio data is optimizedaccording to the reference-timing information.

Audio information accompanying video information may be replaced bycharacter information to be indicated on a multiplexed basis for acommentary, figure information, picture information different from thevideo information, auxiliary information relating to an indication areaof the video information, or other information to be played backsynchronously with the video information.

As understood from the previous description, regarding the generation ofmain data, it is supposed that a data piece related to the type ofafter-recording-purpose audio data is previously produced in elementdata, and that the transmission rate of the after-recording-purposeaudio data is equal to that of original audio data in the main data.

As previously mentioned, a video signal is compressively encoded into anencoding-resultant video signal. A first audio signal which accompaniesthe video signal is compressively encoded into a firstencoding-resultant audio signal. The encoding-resultant video signal andthe first encoding-resultant audio signal are multiplexed into amultiplexing-resultant signal to which timing information indicative ofdecoding timings and playback timings are added. Themultiplexing-resultant signal is recorded on an information-signalrecording medium. A second audio signal (an after-recording-purposeaudio signal) which has a synchronized relation with the video signal iscompressively encoded into a second encoding-resultant audio signal towhich timing information for the synchronization with the video signalis added. The second encoding-resultant audio signal is recorded on therecording medium. Thereafter, the second audio signal and the videosignal in the multiplexing-resultant signal are synchronouslyreproduced. The second encoding-resultant audio signal is equivalent intiming to the first encoding-resultant audio signal. Thus, during thereproduction of the second audio signal, the decoding of the secondencoding-resultant audio signal is prevented from causing a trouble.

A first multiplexing-resultant signal is defined as an original datastream containing the compressively-encoded versions of a first audiosignal and a video signal. The first audio signal accompanies the videosignal. A second multiplexing-resultant signal is defined as a datastream containing the compressively-encoded versions of a second audiosignal (an after-recording-purpose audio signal) and the video signal.The second audio signal accompanies the video signal. Although thesecond multiplexing-resultant signal is not generated in fact, an AVsignal composed of the second audio signal and the video signal can beplayed back. Since the second multiplexing-resultant signal is neithergenerated nor recorded, the first multiplexing-resultant signal recordedon an information-signal recording medium can be prevented from beingerased by the recording of the second multiplexing-resultant signal.Since the second multiplexing-resultant signal is not recorded, theusable capacity of the recording medium is prevented from being reducedby the second multiplexing-resultant signal.

A plurality of after-recording-purpose audio signals can be recorded onan information-signal recording medium. During playback, a user canselect one from the after-recording-purpose audio signals. In this case,the selected after-recording-purpose audio signal and a video signalaccompanied therewith are synchronously reproduced. Furthermore, thestorage capacity of the recording medium can be efficiently used.

The compressively encoding of a video signal and an audio signal isaccorded with the MPEG1 standards or the MPEG2 standards. The generationof an AV multiplexing-resultant signal is accorded with the MPEG systemstandards. An AV multiplexing-resultant signal may contain videoinformation representing a music program and audio informationrepresenting karaoke music in the music program. In an AVmultiplexing-resultant signal, audio information may be replaced bysecond video information to be played back synchronously with main videoinformation.

The compressively encoding and the systematizing (the multiplexing) areaccorded with the MPEG1 standards, the MPEG2 standards, and the MPEG4standards. The compressively encoding and the systematizing (themultiplexing) may be based on a fractal encoding procedure or anotherencoding procedure in which the signal processing for playback isimplemented while an encoding-resultant signal is temporarily stored ina prescribed recording area.

The information-signal recording medium 25 includes, for example, a harddisk or a DVD-RAM. The recording medium 25 may be of another type whichhas a random access performance enabling two different information filesto be alternately read out, and which has a storage capacity enough torecord a prescribed amount of digital video data.

A signal produced by the information-signal generating apparatus 10 or10 a is recorded on the information-signal recording medium 25. Theinformation-signal reproducing apparatus 30 or 30 a reproduces therecorded signal from the recording medium 25. The reproduced signal canbe transmitted by the information-signal transmission apparatus 30 b.Generally, the recording medium 25 is connected with theinformation-signal generating apparatus 10 or 10 a, theinformation-signal reproducing apparatus 30 or 30 a, and theinformation-signal transmission apparatus 30 b in a manner such thatsignals can be directly transferred thereamong. Alternatively, therecording medium 25 may be connected with the information-signalgenerating apparatus 10 or 10 a, the information-signal reproducingapparatus 30 or 30 a, and the information-signal transmission apparatus30 b via a communication means such as a communication network.

Second Embodiment

A second embodiment of this invention is similar to the first embodimentthereof except for design changes mentioned later. According to thefirst embodiment of this invention, regarding the generation of maindata, it is supposed that a data piece related to the type ofafter-recording-purpose audio data is previously produced in elementdata, and that the transmission rate of the after-recording-purposeaudio data is equal to that of original audio data in the main data.According to the second embodiment of this invention, regarding thegeneration of main data, element data equal in transmission rate toexpected after-recording-purpose audio data are previously recorded on amultiplexed basis as dummy data.

Third Embodiment

A third embodiment of this invention is similar to the first embodimentthereof except for design changes mentioned later. According to thethird embodiment of this invention, in the case whereafter-recording-purpose audio data are expected to be recorded, elementdata equal in transmission rate to the expected after-recording-purposeaudio data are previously recorded on a multiplexed basis as dummy data.When actual after-recording-purpose audio data are generated, thepreviously-recorded dummy data are replaced with the actualafter-recording-purpose audio data. During the replacement of thepreviously-recorded dummy data with the actual after-recording-purposeaudio data, it is unnecessary to alter every PCR, every DTS, and everyPTS.

In the case where the transmission rate of actualafter-recording-purpose audio data differs from the expectedtransmission rate, video data and the actual after-recording-purposeaudio data are recorded on a multiplexed basis. In this case, it isnecessary to renew every PCR, every DTS, and every PTS.

ADVANTAGES PROVIDED BY EMBODIMENTS

The first, second, and third embodiments of this invention provide thefollowing advantages.

A video signal, a first audio signal, and timing information forsynchronous reproduction of video and audio are multiplexed into an AVmultiplexing-resultant signal to be recorded. In addition, acompressively-encoded version of a second audio signal which issynchronous with the video signal is supplied as a signal to berecorded. A reproducing side can reproduce an AV signal composed of thevideo signal and the second audio signal accompanying the video signalwithout being supplied with an AV multiplexing-resultant signalcontaining the video signal and the compressively-encoded version of thesecond audio signal.

An AV multiplexing-resultant signal is a stream of packets to whichtiming information is added. Also, a compressively-encoded version of asecond audio signal is a stream of packets to which timing informationis added. Therefore, management of timings is easy. A reproducing sidecan reproduce an AV signal composed of the video signal and the secondaudio signal accompanying the video signal.

A plurality of second audio signals to which identification signals areadded are defined as substitution signals. During playback, one can beselected from the second audio signals, and the selected second audiosignal can be reproduced.

A video signal, a first audio signal, and timing information forsynchronous reproduction of video and audio are multiplexed into an AVmultiplexing-resultant signal. A compressively-encoded version of asecond audio signal has a synchronized relation with the video signal.The AV multiplexing-resultant signal and the compressively-encodedversion of the second audio signal are reproduced, and the video signalis reproduced from the AV multiplexing-resultant signal. The reproducedvideo signal and the reproduced compressively-encoded version of thesecond audio signal are supplied as reproduced output signalssynchronous with each other.

One is selected from compressively-encoded versions of second audiosignals to which identification signals are added. The selectedcompressively-encoded version of the second audio signal can be suppliedas a reproduced output signal having a synchronized relation with avideo signal.

A video signal, a first audio signal, and timing information forsynchronous reproduction of video and audio are multiplexed into an AVmultiplexing-resultant signal. A compressively-encoded version of asecond audio signal has a synchronized relation with the video signal.The AV multiplexing-resultant signal and the compressively-encodedversion of the second audio signal are reproduced, and the video signalis reproduced from the AV multiplexing-resultant signal. The reproducedvideo signal and the reproduced compressively-encoded version of thesecond audio signal are multiplexed into a new AV multiplexing-resultantsignal. The new AV multiplexing-resultant signal is transmitted to areproducing side via a transmission line.

1-12. (canceled)
 13. An apparatus for reproducing an information signal,comprising: means for reproducing an audio-visual multiplexing-resultantsignal generated by multiplexing 1) a compressively-encoding-resultantvideo signal which results from compressively encoding a video signal,2) a first compressively-encoding-resultant audio signal which resultsfrom compressively encoding a first audio signal having a synchronizedrelation with the video signal, 3) a reference clock information signalrelating to synchronization between the compressively-encoding-resultantvideo signal and the first compressively-encoding-resultant audiosignal, and 4) an indication timing information signal based on thereference clock information signal; means for reproducing the videosignal from the reproduced audio-visual multiplexing-resultant signal;means for reproducing a substitution playback audio signal to reproducea second audio signal, the substitution playback audio signal beinggenerated in a signal generation side by 1) compressively encoding thesecond audio signal to get a second compressively-encoding-resultantaudio signal, and 2) adding a sound-production-timing information signalto the second compressively-encoding-resultant audio signal, thesound-production-timing information signal designating a soundproduction timing of the second compressively-encoding-resultant audiosignal, wherein information of a link between the audio-visualmultiplexing-resultant signal and the substitution playback audio signalis generated in the signal generation side; means for deriving thereference clock information signal and the indication timing informationsignal from the reproduced audio-visual multiplexing-resultant signal;means for deriving the compressively-encoding-resultant video signalfrom the reproduced audio-visual multiplexing-resultant signal; meansfor decoding the derived compressively-encoding-resultant video signalto get a decoding-resultant video signal; means for supplying thedecoding-resultant video signal as an output video signal in response tothe derived reference clock information signal and the derivedindication timing information signal; means for deriving thesound-production-timing information signal and the second audio signalfrom the reproduced substitution playback audio signal; and means forsupplying the derived second audio signal instead of audio data in thereproduced audio-visual multiplexing-resultant signal as an output audiosignal in response to the derived reference clock information signal andthe derived sound-production-timing information signal.
 14. An apparatusas recited in claim 13, wherein the substitution playback audio signalcomprises a plurality of subordinate substitution playback audio signalswhich result from compressively encoding source audio signalsrespectively, and different identification signals are added to thesubordinate substitution playback audio signals respectively, andwherein one of the source audio signals is selected by referring to theidentification signals, and the selected source audio signal and thevideo signal are supplied as output reproduced signals synchronous witheach other.